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Abstract — In this paper, we present a new approach for 
the analysis of iterative node-based verification-based (NB-VB) 
recovery algorithms in the context of compressive sensing. 
These algorithms are particularly interesting due to their low 
complexity (linear in the signal dimension n). The asymptotic 
analysis predicts the fraction of unverified signal elements at 
each iteration i in the asymptotic regime where n — >• oo. The 
analysis is similar in nature to the well-known density evolution 
technique commonly used to analyze iterative decoding algo- 
rithms. To perform the analysis, a message-passing interpretation 
of NB-VB algorithms is provided. This interpretation lacks the 
extrinsic nature of standard message-passing algorithms to which 
density evolution is usually applied. This requires a number 
of non-trivial modifications in the analysis. The analysis tracks 
the average performance of the recovery algorithms over the 
ensembles of input signals and sensing matrices as a function 
of I. Concentration results are devised to demonstrate that 
the performance of the recovery algorithms applied to any 
choice of the input signal over any realization of the sensing 
matrix follows the deterministic results of the analysis closely. 
Simulation results are also provided which demonstrate that 
the proposed asymptotic analysis matches the performance of 
recovery algorithms for large but finite values of n. Compared 
to the existing technique for the analysis of NB-VB algorithms, 
which is based on numerically solving a large system of coupled 
differential equations, the proposed method is much simpler and 
more accurate. 

I. Introduction 

Noiseless compressed sensing was introduced with the idea 
to represent a signal v g R" with k non-zero elements using 
measurements c e R m , where k < m <C n, and yet be able 
to recover the original signal v back QJ, (2). In the mea- 
suring process, also referred to as encoding, signal elements 
are mapped to measurements through a linear transformation 
represented by the matrix multiplication c = Gv, where the 
matrix G € R" IX ™ is referred to as the sensing matrix. This 
linear mapping can also be characterized by a bipartite graph 
||3), referred to as the sensing graph. 

In the recovery process, also referred to as decoding, based 
on the knowledge of the measurements and the sensing matrix, 
we estimate the original signal. The decoding process is 
successful if v is estimated correctly. Three performance 
measures namely, density ratio 7 = k/n, compression ratio 
T"c — n/m, and oversampling ratio r Q — m/k are used in 
order to measure and compare the performance of the recovery 
algorithms in the context of compressed sensing^ 

'in 14) authors proved that the lower bound r a = 1 (7 = l/r c ) is 
achievable in the asymptotic case (n — > 00). Indeed, a decoder based on the £0 
norm can achieve ro ~ 1 under certain conditions [5]. Despite its conspicuous 
performance, the solution to the to recovery is numerically unstable |6j. 



Perhaps one of the most interesting recovery algorithms 
in compressed sensing, and also a suitable choice for low- 
complexity encoding/decoding purposes, is a sub-class of 
iterative message-passing algorithms called Verification-Based 
(VB) algorithms. The two algorithms in this class were in- 
troduced in the context of channel coding over non-binary 
alphabet Q, adopted in the context of compressed sensing 
|8|, [9], and studied in numerous literature (3), (7j-(T3]. For 
each VB algorithm two descriptions exist, which are known 
as node-based (NB) and message- based (MB) j9j, (lO|0The 
VB algorithms are well-known for their low computational 
complexity, 0(n), and their insensitivity to the distribution 
of non-zero entries in both the sensing matrix and the in- 
put signal (under certain conditions). These algorithms are, 
however, expected to be sensitive to the presence of noise in 
the measurements. Therefore, the performance of noise-free 
algorithms can serve as an upper bound for the performance 
of the noisy versions in real-life applications. 

Authors in (7), (9)> (TOj demonstrated that for each VB 
algorithm as n — > 00, there exists a limiting value for 7, before 
which the recovery algorithm is successful with probability 
one. Researchers in J7J, (9), fT0| analyzed VB algorithms in 
the asymptotic case to calculate this limiting value, henceforth 
referred to as success threshold, and then use it to compare 
different VB algorithms and estimate their performance for 
finite (large) n. The NB and MB descriptions yield different 
success thresholds^] and are analyzed using different tools. In 
general, NB descriptions have higher success thresholds and 
are harder to analyze. Asymptotic analysis considered in (7J, 
|9| are of MB type, while authors in [ 1 1 considered the NB 
type for the recovery algorithms. In this paper, we general- 
ize/revise the density evolution analysis | [T5| and propose a 
low-complexity analysis for the NB descriptions. 

II. Previous Works and Motivation 

Density evolution fT5) is a well-known method to analyze 
iterative message-passing algorithms in coding theory. In this 
method, the probability of some events of interest are calcu- 
lated and tracked with iteration number. In the case of MB 
decoders, the calculation and evolution of such probabilities 
follow a closed form solution discussed in (7). Due to the 
fundamental difference between the NB and MB decoders, 

2 In the context of turbo codes, NB and MB approaches are known as non- 
extrinsic and extrinsic message-passing, respectively 1141. 

3 For one of the VB algorithms, the two descriptions perform the same, 
while for the other one, the NB version outperforms the MB one |10| . 



the existing density evolution methodology can not be used 
to analyze the performance of NB algorithms fTO] . For this 
reason, authors in [ 1 1 used techniques based on differential 
equations for formulating the density evolution for the NB ver- 
sions. Due to the large number of resulting coupled differential 
equations, the authors used numerical methods to evaluate the 
performance of the NB algorithms. The problems associated 
with their approach are: 1) since the approximation depends 
on the choice of n, and the analysis is valid for n — > oo, one 
has to choose very large n, which directly translates to long 
running time and high computational complexity, and 2) the 
errors in the numerical approximations, potentially, propagate 
through iterations. Hence, there is no guarantee on the reported 
success threshold by this analysis, even for large n, as our 
numerical results show in Section [Vll 

In this paper we first define the ensemble of graphs and 
inputs of interest as well as the state parameters that fully 
describe the state of the NB-VB decoders at any iteration. 
Then, similar to fl5) , we show that for a given decoder, at 
any iteration, the state parameters associated with almost all 
realizations of graphs and inputs are concentrated around the 
state parameters calculated using a deterministic algorithm 
(also referred to as the density evolution analysis). Using 
the density evolution analysis, we can estimate the success 
threshold of different VB algorithms over a variety of sparse 
graphs. The proposed analysis in this paper assumes noiseless 
measurements and has a computational complexity linear in 
the number of iterations. Compared to the differential equation 
analysis in [ fTO] , our approach uses almost the same number 
of state parameters but relies on addition and multiplication 
operations resulting in substantially less complexity. 

III. Background 
A. Ensembles of Sensing Graphs and Inputs 

A {d v ,d c )-regular bipartite weighted graph (or weighted 
biregular graph) Q(V U C,W(E)) is a graph whose set of 
vertices VUC can be divided into two disjoint sets V and C, 
so that every (weighted) edge in the set E connects a vertex in 
V to one in C. Moreover, each vertex in V (C) is incident to 
d v (d c ) vertices in C (V), where d v and d c are fixed positive 
integers. Following the terminology of graph codes, we refer 
to the sets V and C as the variable nodes and check nodes, 
respectively. To each such graph, corresponds a biadjacency 
matrix A(Q) of size |C| x \V\ formed as follows: assuming an 
arbitrary labeling for nodes in V and C, if there exists an edge 
between the vertices Cj G C and Vj £ V, then the entry 
in the matrix is the weight of such edge, otherwise, a^j = 0. 
Any such biadjacency matrix has n — d c 0's in each row and 
m — d v 0's in each column. 

For given parameters d v ,d c and n (m — nd v /d c ), let 
VVJ(d v ,d c ) denote the ensemble of biadjacency matrices 
of interest corresponding to all (d Vl d c )- weighted biregular 
graphs with n variable nodes, whose weights w are drawn 
i.i.d. according to a distribution f(w). 

To discuss the ensemble of inputs of interest, let a € [0, 1] 
be a fixed real number and v be a vector of length n with 
elements v t drawn i.i.d. according to a mixed probability 



distribution function defined as follows: the element is zero 
with probability 1 — a, or follows a distribution g with 
probability a. We denote the ensemble of all such vectors 
by Vl\a). 

In this paper we consider weighted biregular sensing 
graphsQWe map the sets of signal elements and measurements 
to the vertex sets V and C (\V\ = n, \C\ — m), respectively. 
We will interchangeably use the terms variable nodes and 
signal elements as well as check nodes and measurements. 
Also, the sensing matrix G is regarded as a biadjacency matrix 
drawn at random from the ensemble WJ(d v , d c ). Moreover, 
we draw the signal vector v at random from the ensemble 

B. VB Algorithms and Verification Rules 

The first VB algorithm discussed in this paper, referred to 
as LM, is the LM1 algorithm in |9j. The second algorithm, 
referred to as SBB, is the "sudocode" algorithm in (8), which 
is the same as the LM2 algorithm in |9|. In these algorithms, a 
variable node can be either: "verified" or "unverified". Based 
on the following three verification rules, variable nodes are 
verified and certain values are assigned to them. These values 
remain unchanged throughout subsequent iterations. 

• Zero Check Node (ZCN): If a check node has a value 
equal to zero, all its neighboring variable nodes are 
verified with a value equal to zero. 

• Degree One Check Node (DOCN): If a check node 
is connected to only one unverified variable node, the 
variable node is verified with the value of the check node. 

• Equal Check Nodes (ECN): If N check nodes have the 
same non-zero value then 1) all variable nodes neighbor 
to a subset (not all) of such check nodes are verified with 
the value zero, and 2) if there exists a unique variable 
node neighbor to all N check nodes, then it is verified 
with the common value of the check nodes. 

The SBB algorithm applies all verification rules, while the 
LM algorithm applies all but the ECN rule. In this (non 
message-passing) description, when a variable node is verified 
at an iteration, its verified value is subtracted from the value 
of its neighboring check nodes, and then, removed from the 
sensing bigraph along with all its adjacent edges. Verification 
of variable nodes affects the structure of the graph as well as 
the value of check nodes, which in turn trigger the verification 
of other variable nodes in future iterations. The case in which 
the assigned value to a variable node is different from its 
true value, is called false verification. If at least one of 
the distributions / or g is continuous, then false verification 
happens with probability zero JSJ, j9j, |11|. Henceforth, we 
assume that the probability of false verification for a variable 
node in an iteration is zero. Using the Boole's inequality 
successively, it is easy to show that the probability of false 
verification in the whole recovery process is also zero. 

4 It is part of our ongoing research to extend the results of this paper 
to (weighted) irregular graphs. Our preliminary simulations show that the 
success threshold associated with VB algorithms can be increased by the use 
of carefully designed irregular graphs. 



IV. Message-Passing (MP) Interpretation of VB 
Recovery Algorithms 

A. Definitions and Setup 

A message-passing algorithm works in iterations through 
exchanging messages (belonging to certain alphabets) between 
check nodes and variable nodes along the edges in the graph, 
and processing the received messages at nodes by applying 
specific mapping functions. For the ease of presentation, we 
discuss the message alphabets and mapping functions only 
for the case in which the non-zero weights of the sensing 
graph are drawn from an uncountable or countably infinite 
alphabet set. For other sensing graphs, these parameters should 
be redesigned in order to have message-passing algorithms 
equivalent to the original algorithms discussed before. 

Any message sent from a variable node belongs to an 
alphabet A4 : {0, 1} x K. The first coordinate, called status 
flag, indicates the verification status of the variable node. 
The second coordinate represents the verified value of the 
variable node and is valid only if the status flag is 1. Any 
message sent from a check node belongs to an alphabet 
O : Z x M. The first coordinate indicates the number of 
unverified variable nodes neighbor to the check node. The 
second coordinate indicates the result of the linear combination 
of the unverified neighboring variable nodes. Moreover, the 
edges in NB-MP algorithms, multiply (divide) the second 
coordinate of a message by their associated weight, if it is 
sent from (received by) a variable node. Since the weights are 
chosen independently, all messages are independent. 

Any iteration I > 1, consists of two rounds. A round 
starts with check nodes processing the received messages, then 
proceeds with the transmission of messages from check nodes 
to variable nodes, continues by variable nodes processing the 
received messages, and ends with the transmission of messages 
from variable nodes to check nodes. The two rounds in each 
iteration follow the same procedure and only differ in the 
mapping functions associated to variable nodes. 

For the VB algorithms under consideration, the mapping 
functions in the variable nodes and check nodes are not 
functions of the iteration number. Hence, let ^i 1 , $•{> : 
O dv — > A4, represent the functions used at any unverified 
variable node to map the incoming messages to the outgoing 
message in the first and second round of any iteration, respec- 
tively. When a variable node becomes verified at an iteration, 
its outgoing message remains unchanged, irrespective of its 
incoming messages. In contrast to the variable nodes, the 
mapping function used in check nodes is identical for both 
first and second round of each iteration. Every check node 
i,i £ {1,2, ••■ , to} = [to] has an associated received 
measurement Cj, a random variable taking values in R. So, we 
use the notation $ c : K x A4 dc — >• O, to denote the mapping 
function used in all check nodes at any iteration. 

B. Recovery Algorithms Based on Message-Passing 

Here we define the mapping functions and <£> c . 

Function embeds the aforementioned verification rules 
DOCN and ECN, while function $[ 2) embeds the ZCN rule. 



Let the message o £ O be an ordered pair of elements (d, c), 
where deZ + ,ceR, and the message m e X be an ordered 
pair of elements (s,v), where s £ {0,1}, v £ K. We also 
assume an arbitrary numbering for edges adjacent to a node 
(either variable node or check node), and use the notation 
Oi,i £ [d v ] and m 3 , j £ [d c \. 

We define the mapping functions <& V X \ <& v 2 ^ embedding all 
three verification rules, discussed before. 

(l,Cj) 3i € [d v ] : o 

*W(oi,--- ,o dv ) = 



(l,Ci) 

(l,c) 3i,j £ [d v ],i ^ j : 

Oi = (di,c),Oj = (dj, 
(0, 0) Otherwise 



c) 



$i 2) (oi,--- ,o dv ) 



(1,0) 3i£ [d v ] :o i = (d u 0) 
(0, 0) Otherwise 



A variable node may not be verified to different values 
according to verification rules above for the same reasons that 
the probability of false verification is zero. 

For any iteration £ > 1, the mapping function at any check 
node is as follows: 

d c d c 

$ c (ci,rai, • • • ,m dc ) = (d c - s t ,Ci - ^SjVi), 



where, is the measurement associated with the check node 
Ci, and m,i = (si,Vi) is the message received along the ith 
edge. Since at iteration zero there is no incoming messages 
from variable nodes to check nodes, we have: 



{d c ,Ci). 



V. Asymptotic Analysis Framework 

We first show that for given probability distribution func- 
tions /, <?, and given constant parameters d v , d c and a (i) the 
performance of a VB algorithm over realizations of the sensing 
graph and the input signal concentrates around the average 
performance of the algorithm, when the average is taken over 
all the elements in the ensemble WJ(d v ,d c ) x V™(a), and 
(ii) for n — > oo, the average performance converges to the 
performance of the VB algorithm over a cycle-free graph 
defined as follows Let ACf, referred to as the neighborhood 
of node v of depth 21, be the subgraph consisting of the 
variable node v and all nodes (either variable node or check 
node) that are connected to v with any path of length less 
than 2£. A graph is cycle-free up to depth 21 when for every 
v, Afy 1 is a tree. 

A. Concentration and Convergence to Cycle-Free Case 

Let be the fraction of variable to check node messages 
with unverified status in a general sensing graph at iteration 
£. Further, let E[/3( £ )] denote the expected value of f}™>, 
where the expectation is taken over all sensing graphs and 
input signals. Let a^' be the expected number of messages 

5 Our general method of proof is similar to that of |15| . However, some 
non-trivial modifications are made since the message-passing interpretation 
of NB-VB algorithms lacks the extrinsic nature of standard message-passing 
algorithms of |15| . 



with unverified status passed along an edge emanating from 
a variable node with a tree-like neighborhood of depth at 
least 2£ at the Ah iteration, where the expectation is taken 
over all weights and all input signals. The following theorem 
shows that over all realizations, f3^ does not deviate far from 
E[/3^], which itself, is not far from as n — > oo, with 
high probability. 

Theorem 1. Over the probability space of all weighted graphs 
Wf(d v ,d c ), and all signal inputs V"(a), for fixed £, letting 
fl( > and a^' be defined as above, there exist positive constants 
fj,(d v ,d c ,£) and rj(d v ,d c ,£) r such that (i) for any e > 0, 



Pr /?W - E[0<*>] > e/2 < 2 e - £2 ™^, 
and (ii) for any e > 0, and n > 2rj/e, 



< e/2. 



(1) 



(2) 



Note that combining ([TJ and (|2), the following holds: for 

any e > 0, and n > 2r]/e, 



Pr 



> e/2 



< 2e~ 



Next, we show that can be calculated under the as- 
sumption of cycle-free neighborhood for the SBB recovery 
algorithm using a deterministic procedure. A similar procedure 
exists for the LM algorithm, which is not discussed here due 
to the space limitation. 

B. State Parameters for the Analysis of the SBB Algorithm 

In the analysis of NB-VB algorithms, the state parameters, 
at the beginning of any iteration I, are defined as the proba- 
bility of variable nodes and check nodes belonging to some 
sets denoted by K, {i \ K,f \ TZ^\ A^, and Af}*). For instance, 

described before as one of the state parameters, is indeed 
the probability that a variable node belongs to the set JO®. In 
the following, we define the aforementioned sets for the SBB 
algorithm. For the LM algorithm, the definitions of the sets 
JC?> and A/fJ change slightly. The set K,^ consists of all un- 
verified non-zero variable nodes, while the set A" consists of 
all unverified zero-valued variable nodes. The set 1Z™> includes 
all variable nodes recovered up to iteration £. Hence, KS l >, 
TZ^\ and A^ are disjoint sets spanning the set of all variable 
nodes. We divide the set of all check nodes into subsets 
where i and j indicate the number of neighboring variable 
nodes in the set /C" and A™\ respectively. Further, the set 

includes all variable nodes in KS® with i neighboring 

check nodes in the set Af[ := 1^=0* -^l ]■ 

In Theorem [2] below, we introduce the necessary and suffi- 
cient conditions for the verification of variable nodes in K.^ 
in each iteration £ for the SBB algorithm. A similar theorem 
can be proved for the set in each iteration £ for the SBB 
algorithm. Theorems with the same spirit as Theorem [2] can 
be proved also for the LM algorithm. 

Theorem 2. In the first round of any iteration £ in the SBB 
algorithm, a non-zero variable node v 6 KM^ is verified if and 



K^p consists of all variable nodes in the set fdf connected 
to the set Af^. 

Based on Theorem [2] we can calculate the probability that 
a variable node in the set KS' is verified at iteration £, 
a W — a^ e+1 \ as a function of state parameters at the same 
iteration. Indeed, it can be shown that all state parameters at 
iteration £ are functions of the same parameters at iteration 
I — 1. In the analysis we track the evolution of the state pa- 
rameters with iteration. For an initial parameter cr°)(= a), the 
recovery algorithm is called successful if lim^oo a"' = 0, 
and is called unsuccessful if there exists e > 0, such that 
lim^oo > e. The success threshold is then defined as the 
supremum of all a' ', such that the algorithm is successful. 

VI. Simulation Results 

Here, we present simulation results obtained by running the 
NB-VB algorithms over graphs of finite length n. We also 
present results obtained by running the proposed asymptotic 
analysis. The comparison of the results shows that there is a 
good agreement between empirical and analytical results for 
moderately large graphs (n w 10 5 ). 

In all simulations, a signal element (variable node) belongs 
to the support set with probability aS°\ Also, each such 
element has a standard Gaussian distribution. The biregular 
graphs are constructed randomly with no parallel edges and 
all weights equal to one. In each experiment, the sensing graph 
is fixed and each simulation point is generated by averaging 
over 1000 random instances of the input signal. 

For the analytical results, based on the fact that is a 
non-increasing function of iteration number £, we consider the 
following stopping conditions: 

1) Success: a® < lO" 7 . 

2) Failure: > 10~ 7 and - a^-^\ < 10~ 8 . 

To calculate the success threshold, a binary search is per- 
formed until the separation between start and end of the search 
region is less than 10 -5 . 

In Table [I] we have listed the analytical success thresholds 
of the LM and SBB algorithms for graphs with compression 
ration r c = d c /d v = 2 and different d v values. For every 
graph, the SBB algorithm has better performance than LM. 
Also, as we decrease d v , algorithms perform better in terms 
of success threshold and oversampling ratio r Q — d v /ad c ^ln 
fact, among the results presented in Table [j] the application 
of the SBB and LM to (3, 6) graphs results in the lowest 
oversampling ratio of w 1.94 and rj 2.94, respectively. 

TABLE I 

Success Thresholds for different graphs and algorithms for 

FIXED COMPRESSION RATIO r c = 2 



{d v ,d c ) 


(3,6) 


(4,8) 


(5,10) 


(6, 12) 


(7, 14) 


SBB 


0.2574 


0.2394 


0.2179 


0.1992 


0.1835 


LM 


0.1702 


0.1555 


0.1391 


0.1253 


0.1140 



only if it belongs to the set Ui"=2 U K-f < where the set graphs 



'-{(■) 



6 These results are consistent with the results observed for the Belief Prop- 
agation (BP) decoding algorithm of binary LDPC codes based on biregular 



To further investigate the degree of agreement between our 
asymptotic analysis and finite-length simulation results, we 
have presented in Fig. |T| the evolution of with iterations 
I for the SBB algorithm over a (5,6) graph with n = 10 5 . 
Two values of a' ' are selected: one above and one below 
the success threshold, which is 0.3892. The theoretical results 
are shown by solid lines while simulations are presented with 
dashed lines. The two sets of results are in close agreement 
particularly for the cases where a^- ' is above the threshold 
and for smaller values of I. 




15 20 25 

Iteration (t) 



Fig. 1. Evolution of vs. I for the SBB algorithm over a (5,6) graph. 

Next, for different values of a^°\ we estimate the average 
fraction of unverified non-zero variable nodes using the 
analysis, and denote the value of a^ e ' at the time that the 
analysis stops (because one of the stopping criteria is met) 
as a^ stop \ These values are plotted vs. the corresponding 
values of a' ' in Fig. [5] for the SBB algorithm over the (5, 6) 
sensing graphs. In the same figure, we have also given the 
corresponding simulation results for two randomly selected 
(5, 6) sensing graphs with n = 10 5 and 10 6 . The simulation 
results for both lengths closely match the analytical results, 
with those of n = 10 6 being practically identical to the 
analytical results. From the figure, it can also be seen that 
as increases and tends to one, the curves tend to the 
asymptote a ( stop ) = a^°\ 
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Fig. 2. c.( s *°p) vs. c*(°) for SBB applied to (5, 6) graphs. 

As the last experiment, we compare the running time and 
accuracy of the proposed asymptotic analysis against those 



executed on an AMD Phenom 9650 Quad-Core 2.3 GHz CPU 
with 3 GB RAM. The success threshold of 0.2574 is obtained 
in 23.1 seconds. Table |H| summarizes the results of running 
the analysis of [ 10 1 on the same machine for different values 
of n. The reported thresholds increase with the increase in 
n. For n = 100, 000, the running time is roughly 100 times 
that of our proposed method. Moreover, the obtained threshold 
of 0.2591 is only in agreement with the threshold of 0.2574, 
obtained by the proposed method, up to two decimal points. In 
fact, experiments similar to those reported in Fig. [T] reveal that 
the accuracy of the threshold obtained by the method of flO) is 
lower than our results. In particular, our simulations show that 
the SBB algorithm over (3, 6) graphs with n = 10 5 fails for 
a (0) = 0.259, which would imply that the threshold 0.2591 is 
only accurate up to two decimal points. 

TABLE II 

Success threshold and running time of the differential 
equation analysis of (to). 



n 


Success Threshold 


Running Time (seconds) 


1,000 


0.2577 


9.9 


10, 000 


0.2589 


103.9 


20, 000 


0.2590 


220.6 


50, 000 


0.2590 


647.4 


100, 000 


0.2591 


2044.1 



in the differential equation approach presented in [10|. For 
comparison, a biregular (3, 6) graph and the SBB algorithm 
were chosen. The analysis is implemented in MATLAB and 
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